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MAXIMUM ENTROPY DISTRIBUTION OF ORDER STATISTICS WITH 

GIVEN MARGINALS 

CRISTINA BUTUCEA, JEAN-FRANgOIS DELMAS, ANNE DUTFOY, AND RICHARD FISCHER 


Abstract. We consider distributions of ordered random vectors with given one-dimensional 
marginal distributions. We give an elementary necessary and sufficient condition for the 
existence of such a distribution with finite entropy. In this case, we give explicitly the 
density of the unique distribution which achieves the maximal entropy and compute the 
value of its entropy. This density is the unique one which has a product form on its support 
and the given one-dimensional marginals. The proof relies on the study of copulas with 
given one-dimensional marginal distributions for its order statistics. 


1. Introduction 

Order statistics, an almost surely non-decreasing sequence of random variables, have re¬ 
ceived a lot of attention due to the diversity of possible applications. If A = (Ai,..., A^) 
is a d-dimensional random vector, then its order statistics = (A^;^),..., A(^)) corre¬ 

sponds to the permutation of the components of A in the non-decreasing order, so that 
A(i) < A( 2 ) < ■ ■ • < X(d)- The components of the underlying random vector A are usually, 
but not necessarily, independent and identically distributed (i.i.d.). Special attention has 
been given to extreme values A(i) and A(^), the range A(^) — A(i), or the median value. Di¬ 
rect application of the distribution of the fc-th largest order statistic occurs in various fields, 
such as climatology, extreme events, reliability, insurance, financial mathematics. We refer 
to the monographs of David and Nagaraja [7] and Arnold, Balakrishnan, and Nagaraja [1] 
for a general overview on the subject of order statistics. We are interested in the dependence 
structure of order statistics, which has received great attention when the underlying random 
vector is i.i.d. and for the non i.i.d. case as well. In the i.i.d. case, Bickel [3] showed that 
any two order statistics are positively correlated. The copula of the joint distribution of A(i) 
and A(^) is derived in Schmitz [18] with exact formulas for Kendall’s r and Spearman’s p. In 
Averous, Genest, and Kochar [2], it is shown that the dependence of the j-th order statistic 
on the i-th order statistic decreases as the distance between i and j increases according to the 
bivariate monotone regression dependence ordering. The copula connecting the limit distribu¬ 
tion of the two largest order statistics, called bi-extremal copula, is given by de Melo Mendes 
and Sanfins [8] with some additional properties. Exact expressions for Pearson’s correlation 
coefficient, Kendall’s r and Spearman’s p for any two order statistics are obtained in Navarro 
and Balakrishnan [15]. For the non i.i.d. case, Kim and David [13] shows that some pairs of 
order statistics can be negatively correlated, if the underlying random vector is sufficiently 
negatively dependent. Positive dependence measures for two order statistics are considered 
in Boland, Hollander, Joag-Dev, and Kochar [4] when the underlying random variables are 
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independent but arbitrarily distributed or when they are identically distributed but not in¬ 
dependent. A generalization of these results for multivariate dependence properties is given 
by Hu and Chen [10]. See also Dubhashi and Haggstrom [9] for conditional distribution of 
order statistics. 

Here, we focus on the cumulative distribution function (cdf) of order statistics with¬ 
out referring to an underlying distribution. That is, we consider random vectors X = 
(Ai,... ,Xd) G such that a.s. Ai < • • • < X^ and we suppose that the one-dimensional 
marginal distributions F = (Fj, 1 < i < d) are given, where Fj is the cdf of Aj. A neces¬ 
sary and sufficient condition for the existence of a joint distribution of order statistics with 
one-dimensional marginals F is that they are stochastically ordered, that is: 

(1) Fj_i(x) > Fj(x) for all 2 < i < d,X G M. 

With the marginals fixed, the joint distribution of the order statistics can be characterized 
by the connecting copula of the random vector, which contains all information on the depen¬ 
dence structure of the order statistics. Copulas of order statistics derived from an underlying 
i.i.d. sample were considered in [2] in order to calculate measures of concordance between 
any two pairs of order statistics. For order statistics derived from a general parent distri¬ 
bution, Navarro and Spizzichino [16] shows that the copula of the order statistics depends 
on the marginals and the copula of the parent distribution through an exchangeable copula 
and the average of the marginals. Construction of some copula of order statistics with given 
marginals were given in Lebrun and Dutfoy [14]. 

Our aim is to find the cdf of order statistics of dimension d with fixed marginals which 
maximizes the differential entropy H defined as, for a cdf F with density /: 

H{F) = - I /log/, 

and H{F) = —oo if F does not have a density. If Z is a random variable with cdf F, we shall 
use the convention H{Z) = H{F). In an information-theoretic interpretation, the maximum 
entropy distribution is the least informative among order statistics with given marginals. This 
problem appears in models where the one-dimensional marginals are well known (either from 
different experimentation or from physical models) but the dependence structure is unknown. 
In Butucea, Delmas, Dutfoy, and Fischer [6], we gave, when it exists, the maximum entropy 
distribution of (Ai,..., A^) such that Aj is uniformly distributed on [0,1] for 1 < i < d and 
the distribution of A(^) = maxi<j<rf Aj is given. 

For a d-dimensional random variable A = (Ai,..., A^;) with cdf F and copula Cp, the 
entropy of F can be decomposed into the sum of the entropy of its one-dimensional marginals 
plus the entropy of Cp (see Zhao and Lin [19]): 

d 

H{F) = Y,H{¥,) + H{Cp), 

i=l 

where Fj is the cdf of Aj. In our case, since the marginals F = (Fj, 1 < i < d) are fixed, 
maximizing the entropy of the joint distribution F of an order statistics is equivalent to 
maximizing the entropy of its copula Cj?. Therefore we shall find the maximum entropy copula 
for order statistics with fixed marginal distributions. To solve this question, we introduce the 
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functional of F satisfying (1): 



Fi{dt) |log(F,_i(t)-Fi(t))|. 


The main result of this paper is given by Theorem 5.4 which we reproduce here. 


Theorem. Let F = (Fj, 1 < i < d) be a d-dimensional vector of cdf’s on M satisfying ( 1 ). 

• If there exists 1 <i < d such that H{Fi) = —oo, or i/ J(F) = +oo, then for all cdf F 
of order statistic with one-dimensional marginals F, we have H{F) = —oo. 

• If H(Fi) > —oo for all 1 < i < d, and J(F) < +oo, then there exists a unique cdf Fp 
of an order statistic with one-dimensional marginals F such that H{Ff) > —oo and 
H{Ff) > 11(F) for any cdf F of order statistic with one-dimensional marginals F. 
Furthermore we have: 

d 

H{F^) = d - I + Y,H{Fi) - 

i=l 


and Fp has density /p given for x = (xi,..., Xd) G by: 
d 


/p(x) = fi(xi)]J 


fiiXi) 


.^2 ^*-1 (*») “ Fi ( xi ) 


exp 




Fi_i(s) -Fi(s) 


ds 1 ^f(x), 




where fj is the density function of Fj and C is the set of ordered vectors 
(xi,..., Xd), that is xi < ■ ■ ■ < Xd, such that Fj_i(t) > Fi(t) for all t G (xj_i, Xj) and 
2 < i < d. (Notice that this last condition is automatically fulfilled i/Fj_i > Fj.J 


The function /p may be well defined even if J1(F) = +co and it might even be a density, see 
Remark 5.5. However in this case the entropy of the corresponding cdf is infinite. The density 
/p has a product form on the domain L^: /p(x) i)l^F(x) for X = (xi,... ,Xd) G 

and some measurable functions pi. Corollary 5.7 asserts that /p is the only density which 
has a product form on the domain and whose one-dimensional marginals are given by F. 
This characterization will be used in a forthcoming paper on nonparametric estimation of /p. 


Example. We consider the following example. Let +oo > Ai > • • • > > 0 and for 

1 < i < d let Fj be the cdf of the exponential distribution with mean l/A* and density 
fj(t) = Aj l{i>o}- Notice that Fj_i > F* on (0,+oo), so that = {(xi,...,Xrf) G 
M‘^;0 < xi < ... < Xd}. It is easy to check that JI(F) < +oo with F = (Fj,l < i < d). 
Elementary computations yield that the maximum entropy density of the order statistic 
(Xi, ..., Xd), where Xi has distribution Fj, is given by; 


/p(xi,...,Xrf) = liF(x)Aie-^2"i (l-e-^2"i) ^ A^ v.RA, 


i=2 (1 - e 

where A* = Aj_i — Aj for 1 < i < d + 1 and A^+i = 0. 

In the particular case A* = (d — i -|- 1)A for some A > 0, we get: 

d 

/p(xi,..., Xd) = liF (x) d! A-' e-^"i (1 - e-^"i 




t= 2 (1 - e 


—Xxj 


By considering the change of variable Ui = 1 — e we get the following result. For 
1 < i < d let Fj be the cdf of the /3(l,d — i -|- 1) distribution with density fj(t) = (d — i + 
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( 0 , 1 )(^)- Notice that Fj_i > Fj on (0,1). The maximum entropy density of the 
order statistic {Ui ,..., Ud), where Ui has distribution Fj, is given by: 

d ^ 

n-' 

1=2 ^ 

Elementary computations give H{Ff) = — log((i!) + 2d — {d + 1) 

In order to prove Theorem 5.4, we first introduce the multidiagonal of a copula. For a cop¬ 
ula C on and a random vector U = (I/i,..., Ud) with cdf C, we consider its order statistics 
= (1/(1),..., U(^d)) and define (5(j) the one-dimensional cdf of for 1 < i < d. Then 
the multidiagonal of C is defined as Sc = {S(i), 1 < i < d). This provides a generalization of 
the diagonal section of the copula C which corresponds to the cdf 5(^d) of = ^SLKi<i<d Ui. 
The maximum entropy copula with fixed diagonal section is given in [6]. The necessary and 
sufficient condition for a d-tuple 5 to be the multidiagonal of a (absolutely continuous) copula 
is provided by Jaworski and Rychlik [12]. In order to prove Theorem 5.4, we first establish 
a one-to-one map between the set of copulas of order statistics with given marginals F and 
symmetric copulas with a fixed multidiagonal which only depends on F, see Lemma 
3.4 and Proposition 3.13. Furthermore, this map preserves the absolute continuity of the 
copula, as well as the entropy up to an additive constant, which depends only on the fixed 
one-dimensional marginals F, see Proposition 3.21. Then, under a necessary and sufficient 
condition on the multidiagonal 5, we find the maximum entropy copula with a given general 
multidiagonal 5 and provide an explicit formula for its density, see Theorem 4.7. (Notice 
Theorem 4.7, which is first established to prove Theorem 5.4, can in fact be seen as a special 
case of Theorem 5.4.) The proof relies on the theory of optimization under infinite dimen¬ 
sional constraints developed by Borwein, Lewis, and Nussbaum [5]. 

The rest of the paper is organized as follows. In Section 2, we introduce the basic notations 
and give the definition of the objects used in later parts. Section 3 describes the connection 
between copulas of order statistics with fixed marginals, and symmetric copulas with fixed 
multidiagonals. In Section 4, we determine the maximum entropy copula with fixed multi¬ 
diagonal. Since we obtain a symmetric copula as a result, this is also the maximum entropy 
symmetric copula with fixed multidiagonal. In Section 5, we use the one-to-one map between 
the two sets of copulas established in Section 3 to give the maximum entropy copula of order 
statistics with fixed marginals . We finally obtain the density of the maximum entropy distri¬ 
bution for order statistics with fixed marginals by composing the maximum entropy copula 
with the marginals. Section 6 contains the detailed proofs of Theorem 4.7 and other results 
from Section 4. Section 7 collects the main notations of the paper to facilitate reading. 

2. Notations and definitions 

2.1. Notations in and generalized inverse. For a Borel set A C W^, we write |A| 
for its Lebesgue measure. For x = (xi,... ,Xd) G and y = (yi,... ,yd) G we write 

X < y if Xi < yi for all 1 < i < d. We define minx = minjxi, 1 < i < d} and maxx = 

max{xi, 1 < f < d} for X = (xi,..., Xd) G If J is a real-valued function defined on M, we 
set J(x) = (J(xi),..., J{xd)). We shall consider the following subsets of W^: 

5 = {(xi,..., Xrf) G xi < • • • < Xrf} and A = 5 n 

with I = [0,1]. In what follows, usually x,y will belongs to and s,t to M or I. For a set 

^4 C K, we note by = M \ A its complementary set. 
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If J is a bounded non-decreasing cad-lag function defined on M. Its generalized inverse J~^ 
is given by = inf{s G M; J(s) > t}, for t G M, with the convention that inf0 = -|-oo 

and inf M = —oo. We have for s, t G M: 

(2) J{t) > s 4^ t > J~^{s), J~^ o J[t) < t and J o J~^ o J[t) = J{t). 

We define the set of points where J is increasing on their left: 

(3) = {t G M;« < t J(«) < 

We have: 

(4) dJ = 0 a.e., 

( 5 ) J-\R) C Ig{J) U {±oo} 
and for s G M, t G Ig{J)- 

(6) J{t) < s t < J~^{s) and J~^ o J(t) = t. 

Notice that if J is continuous in addition, then we have for t G J(M): 

(7) J o = t. 

2.2. Cdf and copula. Let X = {Xi,..., X^) be a random vector on R'^. Its cumulative 
distribution function (cdf), denoted by F is defined by: F{x) = P(X < x), x G R'^. The 
corresponding one-dimensional marginals cdf are (T),! < i < d) with Fi{t) = P(Xj < t), 
t G R. The cdf F is called a copula if Xi is uniform on / = [0,1] for all 1 < i < d. (Notice a 
copula is characterized by its values on only.) 

We define the set of cdf on R'^, C the subset of cdf whose one-dimensional 
marginals cdf are continuous, C C the subset of copulas. We set (resp. C°) the subset 
of absolutely continuous cdf (resp. copulas) on R*^. 

Let us define for a cdf F with one-dimensional marginals (T), 1 < i < d) the function Cp 
defined on 

(8) CF{y) = F{F^^{yi),...,F-^{yd)), y = {yi,... ,yd) e . 

If F G C]f, then Cp defined by (8) is a copula thanks to (7). According to Sklar’s theorem, 
F is then completely characterized by its one-dimensional marginals cdf (F), 1 < f < d) and 
the associated copula Cp which contains all information on the dependence: 

(9) F(x) = Cf (Fi(xi),... ,Frf(xd)), X = (xi,... ,Xrf) G R"*. 

Equivalently, if X = (Xi,..., X^) has cdf F, then Cp is the cdf of the random vector: 

(10) (Fi(Xi),...,Frf(Xrf)). 

2.3. Order statistics. For F G Cd, we write Pf the distribution of a random vector X = 
(Xi,..., Xd) with cdf F. A cdf F G is a cdf of order statistics (and we shall say that X is 
a vector of order statistics) if Pf(Xi < X 2 < ... < Xd) = 1. Let us denote by C the 
set of all cdf of order statistics with continuous one-dimensional marginals cdf. The d-tuples 
(Fj, 1 < i < d) of marginal cdf’s then verify Fj_i > Fj for all 2 < f < d . Let Fd be the set 
of d-tuples of continuous one-dimensional cdf’s compatible with the marginals cdf of order 
statistics: 


( 11 ) 


Fd = {F = (Fi, 1 < i < d) G > Fi, V2 < f < d}. 
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For a given F = (Fj, 1 < i < d) in we define the set of cdf’s F of order statistics with 
marginals cdf F: 

(12) = {F G £05. F, = F„ 1 < i < d}. 

If F G Fd, then we have C^^{¥) / 0 , since the cdf of (F^^(C/),... ,F^^(C/)), U uniformly 
distributed on /, belongs to We define C‘^'^(F) the set of copulas of order statistics 

with marginals F: 

(13) C°^(F) = {CirGC;FG/:^^(F)}. 

According to Sklar’s theorem, the map F i— )■ Cp is a bijection between /1^‘^(F) and C‘^'^(F) 
if F G Fd. 


2.4. Entropy. The Shannon-entropy for a cdf F G is given by: 


(14) 


H{F) 


-oo if F G \ 

-4,/log(/) ifFG£0, 


with / the density of F. We will use the notation H{X) = H{F) if A is a random vector with 
cdf F and H{f) = H{F) if F has density /. The entropy of any F G can be decomposed 
into the entropy of the one-dimensional marginals cdf (Fj, 1 < i < d) and the entropy of the 
associated copula Cp, (see [19]): 


(15) 


H{F) = Y,H{F) + H{Cp). 

i=l 


For F = (Fj, 1 < i < d) G Fd, we define J(F) taking values in [0, -|-oo] by: 

d .. 

(16) J(F) = V / YMt) |log(F,_i(t)-F,(t))|. 

i=2 

Our aim is to find the cdf F* G C^^iY) which maximizes the entropy H. We shall see that 
this is possible if and only if J(F) is finite. From an information theory point of view, this 
is the distribution which is the least informative among distributions of order statistics with 
given one-dimensional marginals cdf F. Since the vector of marginal distribution functions F 
is fixed, thanks to (15), we notice that H{F) is maximal on F^'^(F) if and only if H{Cp) is 
maximal on C^'^(F). Therefore we focus on finding the copula C* G C‘^‘®^(F) which maximizes 
the entropy H. We will give the solution of this problem in Section 5 under some additional 
hypotheses on F. 


3. Symmetric copulas with given order statistics 

In this Section, we introduce an operator on the set C‘^‘^(F) of copulas of order statistics 
with fixed marginals cdf F. This operator assigns to a copula C G C^'^(F) the copula of 
the exchangeable random vector associated to the order statistics with marginals cdf F and 
copula C. We show that this operator is a bijection between C^'^(F) and a set of symmetric 
copulas which can be characterized by their multidiagonal, which is a generalization of the 
well-known diagonal section of copulas. This bijection has good properties with respect to 
the entropy H, giving us a problem equivalent to maximizing H on C^'^(F). We shall solve 
this problem in Section 4. 
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3.1. Symmetric copulas. For x = {xi,... ,Xd) G we define x^^ = ...,the 

ordered vector (increasing order) of x, where X(i) < • • • < X((i) and Yl'i=i ^x(i), 

with St the Dirac mass at t G M. 

Let Sd be the set of permutations on {1, ..., d}. For x = (xi,..., Xd) G and tt G Sd, we 
set Xtt = (a^ 7 r(i), • • • A function h defined on is symmetric if h{x.n-) = h{x) for all 

TT G Sd- A random vector X taking values in is exchangeable if is distributed as X for 
all TT G Sd- In particular a random vector X taking values in is exchangeable if and only 
if its cdf is symmetric. Let (resp. denote the subset of Cd (resp. C) of symmetric 

cdf (resp. copulas) on R'^. 

Let F G and define its symmetrization G 

(17) F^y^{x) = ^ E ^ ^ 

TTGSd 

In particular, if A is a random vector taking values in R'^ with cdf F and IT is a random 
variable independent of A, uniformly distributed on Sd, then An is exchangeable with cdf 

psym^ 

We define the following operator on the set of copulas of order statistics. 

Definition 3.1. LetF G (/l}'^)'^. ForC G C we define Sf{C) as the copula of the exchangeable 
random variable An, where X is a random veetor on R*^ with one-dimensional marginals cdf 
F and copula C and 11 is an independent random variable uniform on Sd- 

The application S'p is well-defined on C and takes values in In the above definition, 

with F = (Fj, 1 < i < d), the one-dimensional marginals cdf of An are equal to: 

( 18 ) 

i=l 

Since the one-dimensional marginals cdf Fj are continuous, we get that G is continuous and 
thus the cdf of An belongs to In particular, thanks to Sklar’s theorem, the copula of 
An is indeed uniquely defined. 

Combining (9), (8) and (17), we can give an explicit formula for 5 f(C'); 

(19) 5F(C)(u) = i E 

TT€Sd 

Remark 3.2. The copula 5 'f(C') is not equal in general to the exchangeable copula 
defined similarly to (17) by = {'^/d\)'^^^g^C{xT^). However this is the case if the 

one-dimensional marginals cdf Fj are all equal, in which case Fj = G for all 1 < i < d. 

If A is a random vector on R'^, let X^^ = (A(i),..., A(^)) be the order statistics of A. 
The proof of the next Lemma is elementary. 

Lemma 3.3. Let X be a random vector on with cdf F G Cd and H a random variable 
independent of X, uniformly distributed on Sd- We have: 

• If Fg C^^, then as- (An)^^ = A 

• If F G then (A‘^'^)n has the same distribution as A. 

For F G Fd, we define the set of copulas C^y"^(F) C as the image of C‘^‘^(F) by the 
symmetrizing operator Sf- 

( 20 ) 


c^ym(F) = Sf{c^^{f))- 
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The following Lemma is one of the main result of this section. 

Lemma 3.4. Let F G The symmetrizing operator Sy is a bijection from onto 

C^y^{Y). 

Proof. Let C'i,C '2 G C‘^'^(F) with 5 f(C'i) = S'f(C' 2 )- Let X and Y be random vectors with 
one-dimensional marginals cdf F and copula C'i,C '2 respectively. Since Ci,C 2 G C‘^'^(F), we 
get that X and Y are order statistics. Notice Xjj and Yjj have the same one-dimensional 
marginals according to (18) and same copula given by S'f(C'i) = 5f(C' 2). Therefore Xu and 
In have the same distribution. Thus, their corresponding order statistics and (Ln)*^'^ 

have the same distribution. By Lemma 3.3 we get that X and Y have the same distribution 
as well, which implies Ci = 02 - □ 

Remark 3.5. We have in general C®^™'(F) ^ C‘^‘^(F) One exception being when the 

marginals cdf’s Fj are all equal. In this case, both sides reduce to one copula which is the 
Frechet-Hoeffding upper bound copula: C~^{u) = minrt, u G 

3.2. Multidiagonals and characterization of C®^"^(F). Let C G C be a copula and U a 
random vector with cdf C. The map t i—)■ C{t,... ,t) for t £ I, which is called the diagonal 
section of C, is the cdf of max?7. We shall consider a generalization of the diagonal section 
of C in the next Definition. 

Definition 3.6. Let C £ C be a copula on and U a random vector with cdf C. The 
multidiagonal of the copula C, 5c = (%),1 <i <d), is the d-tuple of the one-dimensional 
marginals cdf of = (D(i),..., U^d)) the order statistics of U: for 1 < i < d 

<t), t£L. 

We denote hy V = {5c'.,C £ C} the set of multidiagonals. Notice that V C Fd, see 
Remark 3.7. For 5 £ V a multidiagonal, we define Cs = {C]5c = 5} the set of copulas with 
multidiagonal 5. 

A characterization of the set F is given by Theorem 1 of [12]: a vector of functions 
5 = (5(1), • • •, 5(^)) belongs to V if and only if 5(j) G Ci and the following conditions hold: 

(21) %)>%+!), l<i<d-l, 

d 

(22) ^5(,)(s) = ds, 0<s<l. 

i=l 

Remark 3.7. The condition (22) implies that 5(j) G 1 < i < d, moreover they are d- 
Lipschitz. Also, it is enough to know d — 1 functions from 5(j), 1 < i < d, the remaining one 
is implicitly defined by (22). Condition (21) along with the continuity of 5(j) implies that 
any multidiagonal 5c is compatible with the continuous marginal distributions of an order 
statistics, therefore V C Fd- 

Remark 3.8. Since 5(j), 1 < i < d are non-decreasing and d-Lipschitz, we have for almost 
every t £ I: 0 < (5(j))'(t) < d and thus |(5(j))'(t) log((5(j))'(t))| < dlog(d) for d > 2. We 
deduce that for d > 2: 


( 23 ) 


|-f7(5(j))| < dlog(d). 
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Remark 3.9. Let C G l^e a symmetric copula on and U a random vector with cdf 
C. We check that the multidiagonal 5c = ((5(i), 1 < i < d) can be expressed in terms of the 
diagonal sections (C'jj} ,1 < i < d) where for 1 < i < d: 

C{i}{t) = P I max Uk < tj = C{t, ..., t, 1,. .., 1), t G I. 

i terms d—i terms 

According to 2.8 of [12], we have for 1 < i < d: 

%)«)=]) < 6 1 . 

J —^ 

Conversely, we can express the functions {C^iy, 1 < i < d) with 5c- For 1 < i < d and t G I, 
we have: 


Cyiy{t) = P ( max Uk < t\ 

V j 

= Uk = U^)] F 

j=i ^ ' 

d p-n 

j=i \i) 

where we used the definition of d(j) and the exchangeability of U for the third equality. 

The next technical Lemma will be used in forthcoming proofs. Recall that J~^ denotes 
the generalized inverse of a non-decreasing function J, see Section 2.1 for its dehnition and 
properties, in particular, J~^ o J{t) <t ioi t G M. Recall also that for x = (xi,..., x^) G 
we write G{x) = (G(xi),..., G{xd))- 

Lemma 3.10. Let X = (Ai,... ,Xd) he a random veetor on with one-dimensional margi¬ 
nals edf {Fi, 1 < i < d) G (Ti)*^. Set G = Yli=i -Fi/d. IFe have for 1 < i < d: 

(24) P(Ai < o G(t)) = P(Ai < t), tGR, that is FioG-^ oG = Ft. 

We also have for x gW^: 

(25) P(G(A) < x) = P(A < G-^x)). 

Proof. Since G is the average of the non-decreasing functions F), if G(s) = G{s') for some 
s,s' G M, then we have Fi(s) = Fi{s') for every 1 < i < d. Thanks to (2), we have 
G o G~^ o G{t) = G{f) and thus Fi o G~^ o G{t) = Fi{t). This gives (24). 

Recall dehnition (3) for Ig{J) the set of points where the function J is increasing on their 
left. Since G is the average of the non-decreasing functions Fi, we deduce that Ig{G) = 
Ui<j<rfdg(F)). Notice that a.s. Xi belongs to Ig{Fi). Thanks to (6), we get that a.s. 
{G(a) < x} = {A < G-^(x)}. This gives (25). □ 

We will also require the following Lemma. 

Lemma 3.11. Let X = (Ai,..., A^) be a random veetor on with one-dimensional margi¬ 
nals cdf {Fi, 1 < i < d) G {L-i)^- Set G = Yli=i Fi/d. We have for 1 < i < d: 

(26) {FioG-^^ = GoF-\ 
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Proof. Recall Definition (3) for Ig{J) the set of points where the function J is increasing on 
their left. Let 1 < i < d. Thanks to (5), we have C Ig{Fi) U {±oo}. Since G is the 

average of the non-decreasing functions Fj, we deduce that Ig{G) = Thus we 

get; 

(27) 4(G) U {Too}, 

for all 1 < i < d. The function FjoG“^ is also bounded, non-decreasing and cad-lag therefore 
we have for t, s, G M: 

t>(FiO G-^)-\s) ^ Fi o G-\t) > s ^ G-\t) >F-\s)^t>Go F-\s), 

where we used the equivalence of (2) for the first and second equivalence, (27) and the 
equivalence of (6) for the last. This gives that (Fj o G“^)“^ = G o F~^. 

□ 

In the following Lemma, we show that for F G F^, all copulas in L^^™(F) share the same 
multidiagonal denoted by 

Lemma 3.12. Let F = (Fj, 1 < i < d) G F^. Let G G C‘^‘^(F) and U be a random veetor 
with edf Sf{G). Let = (d(i),l < i < d) he the multidiagonal of Sf{G), that is the one¬ 
dimensional marginals cdfofU^^, the order statistics ofU. We have that 6^ does not depend 
on G and for 1 < i < d: 

(28) d(i) = Fi o G-i and = G a F'^, 
with G given by (18). Furthermore, G is the unique eopula of . 

With obvious notation, we might simply write = F o G~^, with G given by (18). 

Proof. Let X be a random vector of order statistics with marginals F G F,^ and copula G. 
Then Sf(G) is the copula of the exchangeable random vector Xn, where LI is uniform on 
and independent of X. We have already seen in (18) that the one-dimensional marginals of 
Xn have the same distribution given by G G CY- Thanks to (10), we deduce that the random 
vector U, with cdf 5 f(G), has the same distribution as G(Xn). Since G is non-decreasing, 
this implies that the order statistics of U, , has the same distribution as G ((Xn)*^'^) 
that is as G(X), thanks to Lemma 3.3. Then use (25) to get for x G W^: 

(29) F{U^^ <x) = P(G(X) < x) = P(X < G-^®))- 

This gives the first part of the Lemma as the multidiagonal of U is the one-dimensional 
marginals cdf of its order statistics. The second equation in (28) is due to Lemma 3.11. The 
fact that G is the copula of and its uniqueness are due to (29) and the continuity of (I(i) > 
see Remark 3.7. □ 

According to the next Proposition, (20) and Lemma 3.4, we get the main result of this 
Section: for any F G F^, the symmetrizing operator Sf is a bijection between G^‘^(F) and 

Proposition 3.13. Let F G F^. We have C®^™'(F) = Qf n . 

Proof. By Lemma 3.12, we have C^y^fF) C Qf 

Let G G CsF n and G be a random vector with cdf G. Let G be given by (18). 
Notice that X = G“^(G) is an exchangeable random vector with marginals G and copula 
G. Thanks to Lemma 3.3, the proof will be complete as soon as we prove that the one¬ 
dimensional marginals cdf of = (Xj-^),... ,X((^)), the order statistics of X, is given by 
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F. Notice = G with the order statistics of U whose one-dimensional 

marginals cdf are given by (5^. We have for 1 < i < d and t G M: 

P(X(,) <t)= P(G-i(C/(,)) <t) = nUi^) < G{t)) = Fi O G-i O G{t) = F,(t), 

where we used (2) for the second equality, (28) for the third, and (24) for the last. This 
finishes the proof. □ 

We end this Section by an ancillary result we shall use later. 

Lemma 3.14. Let F G We have J(F) = JI((5^). 

Proof. Let F = (Fj,l < i < d). We get, using (28) and the change of variable s = G~^{t) 
that: 

[ |log (%-i)(i) - %)(t)) \dt= [ Fi{ds) |log (Fi_i(s) - Fi(s))| . 

Jl 4 g - 1 (( 0 , 1 )) 

Since dFi = 0 outside G“^((0,1)) (as G is increasing as soon as F* is increasing), we get the 
above last integration is also over M. We deduce that; 

d „ 

E (F*-i(^) - Fi{s))\ = J(F). 

i=2 

□ 

3.3. Density and entropy of copulas in C®^™(F). We prove in this Section that Sf 
preserves the absolute continuity on C‘^‘^(F) for F G Td and the entropy up to a constant. 
Let us introduce some notation. For marginals F G Xd, let 

(30) Tf = {s G M, Fj_i(s) > Fi(s)} for 2 < i < d. 

The complementary set (4'?)'^ is the collection of the points where Fj_i = Fj. We dehne 
C I as: 

(31) sF = |jF,((4/f)'=). 

i=2 

By Remark 3.7 we have P C Xd, then the definitions (30) and (31) apply for all 5 £ V. In 
particular, for 6 = (d(i), • • •, d(^d)) £ F> the sets 2 < i < d are open subsets of I, therefore 
(\I/^)c n / is a compact subset. This and the continuity of imply that (5(j)(('I'^)'^) = 

'^(0 (('^i )'^ n I) is also compact, hence is compact. Notice that {0,1} C always holds. 

We define Cg = Cs r\ the subset of absolutely continuous copulas with multidiagonal 5 
and the subset = {5 G V, 0 } of multidiagonals of absolutely continuous copulas. 

According to Theorem 2 of [12], the multidiagonal 5 belongs to if and only if it belongs 
to P and the Lebesgue measure of is zero: |S^| =0. 

Lemma 3.15. Let 5 gV. We have 5 G if and only if for all 2 < i < d, a.e.: 

(32) <J(j_i)l(^5)c = = 0. 

Furthermore, we have that J((5) < -|-oo implies 5 G . 
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Proof. Let J be a function defined on I, Lipschitz and non-decreasing. Let A be a Borel 
subset of I. We have: 


= [ ^.JiA){t)dt = [ l{sej-ioj{A)}J'{s)ds= [ lA{s)J'is)ds, 

J Jo Jo 

where we used (4) and (6) for the last equality. This gives that | J(^)| = 0 if and only if a.e. 
J'Ia = 0. Then use that 5 G if and only if |<5(j)(('l'i)'^)| = 0 for all 1 < i < d and that 
d(i_i)(('I'^)‘^) = to conclude that 5 gV^ \i and only if (32) holds for all 2 < i < d. 

The last part of the Lemma is clear. □ 


Definition 3.16. Let C Td be the subset of marginals F such that there exists an abso¬ 
lutely continuous cdf of order statistics with marginals F, that is n / 0. 

In particular, we have C The next Lemma gives a characterization of the set 

Lemma 3.17. Let F G Pd- Then F G P'11 if and only ifFi G £? for 1 <i < d and |S^| = 0. 
Furthermore, we have that Fi G for 1 < i < d and J(F) < -|-oo imply F G P^. 

Proof. Let F G C'^^fF). We know that F G if and only if Fj G £)* for 1 < i < d 
and Cf G C^, the subset of absolutely continuous copulas (see for example [11]). Therefore 
F G if and only if Fj G T? for 1 < i < d and C‘^‘^(F) n / 0. Recall that is defined 

by (28). We first show that 

(33) C^^(F) n C V 0 if and only if n + 0. 

Let C G C‘^‘^(F) nC^. Then Lemma 3.12 ensures that S'f(C') G Qf The absolute 

continuity of S'f(C') is a direct consequence of (19), (28) and Remark 3.7 which ensures that 
, 1 < * < d are d-Lipschitz, therefore their derivatives exist a.e. on /. This ensures that 

c°F n + 0. 

Conversely, let C G Let 17 be a random vector with cdf C. Then its order 

statistics is also absolutely continuous. Therefore the copula of which is 5p^(C') 
by Lemma 3.12, is also absolutely continuous. This proves thanks to Proposition 3.13 and 
Lemma 3.4 that S'p^(C') G C'^'^(F) nC'^. This gives (33). 

Notice that n C^y"^ / 0 is equivalent to / 0, since for any C G we have that 
(jsym (Jefijied by (17) belongs to By Theorem 2 of [12], C^p / 0 if and only if 

has zero Lebesgue measure. The proof is then complete as one can easily verify using (28) 
that and thanks to Lemma 3.14. □ 


From now on we consider F G P^. We give an auxiliary lemma on the support of the 
copulas in C‘^‘^(F) nC°. 

Lemma 3.18. Let F = (Fj,! < i < d) G P^ and C G C'^^fF) nC°. Then the density of C 
vanishes a.e. on I^\T^ with: 

(34) T ^ = {u = (ui, ..., Ud)G I ^-, Ffi(ni) < • • • < F^i(urf)}. 

Proof. Let X = (Xi,..., Xd) be a random vector of order statistics with one-dimensional 
marginals cdf F and copula C G . Let Lf = (C/i,..., Ud) be a random vector with cdf C. 
Then it is distributed as (Fi(Xi),... ,Frf(Xrf)), see (10). We get P(17 G T^) = 1, since X is 
a vector of order statistics and Xi G Ig{Fi) a.s. for 1 < i < d. This gives the result. □ 

Now we establish the connection between the sets C‘^‘^(F) n and C^^”^(F) n C^. 
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Lemma 3.19. Let F G The symmetrizing operator Sy is a bijection from C^^(F) n 
onto nC°. Moreover, if C £ CiC^, with density function c, then the density 

function sf{C) of Sf{C) is given by, for a.e. u = (tti,... ,Ud) £ '■ 


(35) 


sf(C')(ii) 



• • • 


d 

i=\ 


Let be given by (34)- If C £ C^y^(F) DCq with density c, then the density Sp^(C') of 
S'p^(C') is given by, for a.e. u = (iti,... ,Ud) £ I^: 


(36) 


Sp^(C)(u) = d\ 


x-i 

^(1) 




x-i 

’ V) 


n 




Lj'F 


Ml 


/n-i 


S',oS 






Proof. By Proposition 3.13, we deduce that C*2^™-(F) nC° = Lemma 3.4 and the 

proof of Lemma 3.17 ensures that Sp is a bijection between C‘^‘®^(F) n and C*y™-(F) n C^. 
The explicit formula (35) can be obtained by taking the mixed derivative of the right hand 
side of (19). By Lemma 3.18, all the terms in the sum disappear except the one on the right 
hand side of (35). 

To obtain (36), let C £ C^y^(F) n with density c, and U he a random vector with cdf 
C. The order statistics derived from U is also absolutely continuous with cumulative 
distribution function K, and density function k given by; 

k{u) = d\ c{u)l/\{u), u£l'^. 

By Lemma 3.12, 5p^(C') is the copula of . From (8), we have for u = {ui ,..., Ud) £ I‘^. 
(37) 5pi(C)(n) = iL(5(-;(«i),...,5(-;(n,)). 

According to (6), we deduce that G~^ o G o F“^ = F“^ on (0,1). This implies that for 
s,t £ (0,1), 1 < i < j < d: 

<5(-|(s) < 5(-;(t) ^ G O F-n^) < G O F-^t) 

^G-^oGo F-^(s) <G-^oGo F-^(t) 
^F-\.s)<F-\t) 

^GoF-\s) <GoF-\t), 

where we used (28) for the first equivalence, that G~^ is non-decreasing for the first impli¬ 
cation and G is non-decreasing for the second. Thus, we have that for s,t £ (0,1), that the 
two conditions d^^(s) < <^q)(^) and F“^(s) < F~^(t) are equivalent. Thus we deduce that 
the two sets 

and are equal up to a set of zero Lebesgue measure. Then we deduce (36) from (37). □ 


We give a general result on the entropy of an exchangeable random vector and the entropy 
of its order statistics. 


Lemma 3.20. Let X be a random vector on the corresponding order statistics and 

n an independent uniform random variable on Sd. Then we have: 


H{{xO^)n) = \og{d[) + H{xO^). 
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Proof. Let F be the cdf of . If F ^ then the cdf of given by (17) 

verifies also F^^^ ^ therefore F[{{X'^^)u) = H{X^^) + log((i!) = —oo. If F G £)) with 
density function /, then the density function of F®^™ is given by, for x G 

where is the ordered vector of x. Therefore, using that f{x) = 0 if x 7 ^ x^^, we have: 

H{{xo^)n) = -[ fy^iogifyn 

jRd 

= log(d!) - 4 / log(/(x‘^'^)) dx 

d'- jRd 

= log(d!) - / /(x) log(/(x)) dx 

jRy 

= log{d\) + H{X^^). 


□ 


Now we are ready to give the connection between the entropy of C and 5 f(C') for C G 
C‘^‘^(F), which is the main result of this Section. Recall the definition of 5^ = 1 < * < d) 

given in Lemma 3.12 and thanks to Remark 3.8, is finite for all 1 < f < d. 

Proposition 3.21. Let F G Frf and C G C‘^'^(F). Then we have: 

d 

(38) H{Sf{C)) = log(d!) + H{C) + ^ F(d^)). 

i=l 

Proof. Let U be an exchangeable random vector with cdf S'f(C'), and its order statistics. 
According to Lemma 3.12, Lf^^ has one-dimensional marginals cdf = (d^^, 1 < i < d) and 
copula C. Therefore, using (15), we get: 

d 

i=l 

On the other hand, since S'f(C') is symmetric. Lemma 3.3 ensures that {U^^)u has the same 
distribution as U. Therefore Lemma 3.20 gives: 

d 

H{Sf{C)) = H{U) = H {{U^^h) = + log(d!) = H{C) + F(d^)) + log(d!). 

i=l 

□ 


4. Maximum entropy copula with given multidiagonals 

This section is a generalization of [ 6 ], where the maximum entropy copula with given 
diagonal section (i.e. given distribution for the maximum of its marginals) is studied. 

Recall that multidiagonals of copulas on are given by Definition 3.6. We recall some 
further notation: V denotes the set of multidiagonals; for 5 ^ V, Cs denotes the subset of 
copulas with multidiagonal d; denotes the subset copulas which are absolutely continuous, 
and = CsCi C^. The set C V contains all diagonals for which / 0. 
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We give an explicit formula for C* such that H{C*) = max^gc^ H(C), with H the entropy, 
see definition (14). Notice that the maximum can be taken over since the entropy is minus 
infinity otherwise. When d = 2, the problem was solved in [6]. 

Let 5 = l<i<d)GPbea multidiagonal. Since 1 < i < d are d-Lipschitz, the 
entropy of is well defined and finite, see Remark 3.8 and J((5) given by (16) is also 

well defined and belongs to [0, +oo]. 

The next two lemmas provides sets on which the density of a copula with given multidi¬ 
agonal is zero. For d G let: 

(39) Zg = {u ^ there exists 1 < i < d such that d|j)(n(i)) = 0}. 

Lemma 4.1. Let 5 G . Then for all copulas C G with density c, we have clz^ =0 o.e. 

that is c{u)lz^{u) = 0 for a.e. u £ . 

Proof. By definition of we have for all r G /: 

This implies, by the monotone class theorem, that for all measurable subset K of I, we have: 



/ c{u)lK{uu\)du = / 6L(s)ds. 

Jp Jk 

Since c > 0 a.e., we deduce that a.e. c(u)l{y = 0 thus a.e. clz^ =0. □ 

Recall the definition of given by (30) for 2 < i < d. We also define Tf = (0,di) with 
di = inf{s G /;d(i)(s) = 1} and = (ffd+i,!) with gd+i = sup{s G I]5i^d){s) = 0} . Since 

are open subsets of /, there exists at most countably many disjoint intervals {{g^i\d^p), 
j G Ji} such that 

(40) Tf = \J{9^\d^^). 

j&Ji 

We note by + d^p)/2 the midpoint of these intervals for 2 < i < d -|- 1. In 

particular md+i = (1 -|- gd+i)/2. We also define mi = 0. For 5 gV, let: 

(41) Ls = {u = {ui, ...,Ud)G I"*; (u(i_i), U(i)) C T- for all 2 < i < d}. 

We have the following Lemma for all absolutely continuous copula C G with density c. 

Lemma 4.2. Let 5 gV^ and 2 < i < d. Then for all copulas C G with density c, we have 
cli^L^ = 0 a.e., that is for a.e. u = (ui,... ,Ud) G , for all s ^ .■ 

(^2) h- 

Proof. The complementary set ('kf)'^ is given by: 

(43) ('I'l)'=lj{9pLiF)- 

j&Ji 


Let U = {Ui,...,Ud) be a random vector with cdf C G Cg. For 2 < i < d and s G 
that is d(j_i)(s) = d(i)(s), we have: 

P(C/(i_i) < s < C/(i)) = P(t/(i_i) <s)- P([/(i) < s) = %_i)(s) - d(j)(s) = 0. 
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□ 


This implies that (42) holds a.e. for all s G Since Ji is at most countable, 

we have for a.e. u G and for all s G U-.T. that (42) holds. Since for all u G I, 

s ^'i’f there exists s' G \Jj^j^{gi^\d\^^} such that 

we can conclude that for a.e. u G and for all s ^ (42) hold. 

Notice that for all u = (ui,..., u^) G 

d 

(44) li;,^('u) < 

i=l 

We define the function cs on I'^ as, for u = {ui ,..., Ud) G 

1 ^ 

(45) C5 (u) = - {u) JJ at (u(i)), 


i=l 


where the function a^, 1 < i < d, are given by, for t G I: 
(46) ai{t) = K'^{t) 

with for 1 < i < d, t G {g\^\d^p): 

rt 5'(i)(s) 


(47) 


Kiit) = 


ds 


■I'p l)('S) '^(j)('®) 

and the conventions (5(o) = 1 and i^d+i = 0. Notice that for t G Tf: 

(48) it'i(t) =-log(l - 5(i)(t)). 

Remark 4.3. The choice of for the integration lower bound in (47) is arbitrary as any 
other value in {gP,dP) would not change the definition of cs in (45). 

Remark 4.4. For all 1 < i < d, j G Jj, t G {mP,dP), we have the following lower bound for 


m)> L ,— 


d(i-i){dP) - (5(i)(mf0 
%_i)(4^')) - %)(t) 


U)\ 


-j- - ds = log 

%_i)(4^V%)(s) 

Since is non-decreasing and 5(^i_i'^{dP) = 5(^i-^{dP), we have lim^^^y) Ki{t) = -|-oo. 

The following Proposition states that cs is the density of an absolutely continuous sym¬ 
metric copula Cs G n It is more general than the results in [6], where only the 

diagonal S(d) was supposed given. 

Proposition 4.5. Let S G . The function cs defined in (45)-(47) is the density of a 
symmetric copula Cs G In addition, we have: 

d 

H{Cs) = -J(5) + log(d!) + (d - 1) + ^ 77(5(.)). 


i=l 


(49) 
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The proof of this Proposition is given in Section 6.2. The following characterization of Cs 
is proved in Section 6.6. 

Proposition 4.6. Let 5 G . Then Cs is the only copula in whose density is of the form 

(l/d!)lL,(«)nli hi{u(^i-)), where hi, 1 < i < d are measurable non-negative functions defined 
on I. 

The following Theorem states that the unique optimal solution of maxcgC^(4f(C')), if it 
exists, is given by C^. Its proof is given in Sections 6.7 for case (a) and 6.8 for case (b). 

Theorem 4.7. Let S € V. 

(a) //J((5) = +00 then maxcec^ H{C) = —oo. 

(b) If I {6) < +00 then 5 G , maxceQ II{C) > and Cs given in Proposition f.h is 
the unique copula such that H {Cs) = maxcgc^ II{C). 

The copula Cs will be called the maximum entropy copula with given multidiagonal. 


5. Maximum entropy distribution of order statistics with given marginals 


We use the results of Section 4 to compute the density of the maximum entropy copula 
for marginals F G with defined in Section 3.3. Recall 5^ = = F o G ~^ 

and the dehnition of in (31). Recall Ki dehned by (47), for 1 < i < d and defined by 
(34). We dehne the function cf on for u = {ui, ..., u^) G 


(50) 

cf(u) 


(ui)) 


n 

i=2 


d(j_i) o 5r,Hui} - Ui 


fi) 




Recall the function J((5) defined on the set of multidiagonals by (16) and C^f the copula 
with density given by (45)-(47). 


Proposition 5.1. Let F G . The function cf defined by (50) is the density of the copula 
Cf = ‘5p^(C'5f) which belongs to C‘^'^(F). The entropy of Cf is given by: 

(51) H{Cf) = d-l-S{6^). 


Proof. Since F G F)), we have that G According to Proposition 4.5, c^f defined 
by (45) is the density of a symmetric copula C^f which belongs to n C^, thanks 

to Proposition 3.13 and Lemma 3.19. According to Lemma 3.19, formula (36) we get that 
Cf = •Sf^(C' 5 f) is therefore the density of a copula Cf which belongs to C‘^‘®^(F) nC'^. Use 
(36) and (7) to check (50). To conclude, use (38) and (49) to get (51). □ 


Analogously to Lemma 4.2, we have the following restriction on the support of all F G 
£^^{F) Pi Recall the dehnition of in (30). The proof of the next Lemma is similar to 
the proof of Lemma 4.2 and is left to the reader. 

Lemma 5.2. Let F G and t G F, (('kf )'^) for some 2 < i < d. Then we have for all 
F G £^‘^(F) with density function f: 

= 0 for a.e. x = (xi,..., x^) G S. 

For (5 G H, recall the dehnition of Ls in (41). Let L^ = LsCiS. More generally, for F G F^, 
we set: 

(52) = {x = (xi,..., Xd) G S; (xi_i, Xj) C Tf for all 2 < i < d}. 

The next Lemma establishes the connection between the sets L^f dehned by (41) and L^. 
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Lemma 5.3. Let F = (Fi,...,F£;) G with density functions f* for 1 < i < d. Let 
6^ = ((5(1), •••, 5(d)) given by (28), given by (34) and L^f defined by (41)- Then for 
n^=i ^i{xi)dxi .. .Xd-a.e. x we have that 


L'J'F 


(Fi(xi),...,Fd(xd))lL 




x-1 

^(1) 


Fi(xi),...,5r, oF 


= lLF(a;) 


fd) 

Proof. According to (4) and (6), we have ii{t) dt-a.e. that F“^ o Fj(t) = t. This implies that 
Wi=iU{xi) dxi ■ ■ ■ dxd-d^.e., (Fi (xiFd(xd)) belongs to if and only if x G 5. Recall 
the sets given by (30). For 0*=! dxi ■ ■ ■ dxd-a..e. x G 5, we have: 


(5fii o Fi(xi),..., 5..! o Fd(xd)) G Lsf 


-1 


41) 


id ) 


c-1 


O Fj_i(xi_i), O Fi(xi)) C T 


2<i<d 


(j_l) - 

<^\lt G oFi_i(xi_i),5(“)^ oFi(xi)^ : 5(i-i)(t) > 5(i)(t), 2<i<d 

^VtG (GoF-\oFi_i(xi_i),GoF-ioF,(xi)) : F^.i o G-^t) > F^ o G-i(t), 2 <i<d 

G (G(xi_i), G(xi)) : Fi_i o G-\t) > F^ o G-\t), 2 < i < d, 

where the first equivalence comes from the definition of LgF , the second from the definition of 
the third from (28) and the last from the fact that fi{t) dt-SL.e. F~^ oFi(t) = t. Consider 
the change of variable s = G~^{t). We have by (2): 

t < G{xi) G~^{t) < Xi s < Xj. 

Since Xi_i G IgiFi^i) f(_i(xi_i) dxj-i-a.e., we get Xi_i G Ig{G) and by (6): 

G{xi-i) < t Xi_i < G“^(f) Xi_i < s. 

Therefore we deduce that 0*=! ^iixi) dxi ■ ■ ■ dxd-a.e. x G S: 

(5("} o Fi(xi),..., o Fd(xd)) G LgF 
<?=^Vs G (xi_i,Xi) : Fj_i(s) > Fj(s), 2 <i <d 
<^=^x G L^. 


□ 


Using Proposition 3.21, we check that the copula Cp maximizes the entropy over the set 
^05 (f)_ \Yg gg); for X = (xi,..., Xd ) G 

(53) Ff(x) = Cf(Fi(xi), ... ,Fd(xd)). 


Let fj denote the density function of Fj when it exists. Let us further note for 2 < i < d, 
t GR: 


(54) 


m 


m 

Fi_i{t) - Fi{t) 


When the densities f* exist for all 1 < i < d, we define the function /p for x = (xi,..., Xd ) G 
as: 


(55) 


d 

/f(x) = fi(xi) jj£i(xi)exp 
i =2 


r^i 


£i{s) ds 


fXi-l 


1lf{x), 


with L^ given by (52). The next theorem asserts that the cdf Fp maximizes the entropy over 
the set £^'^(F) and that its density is /p. Recall J defined by (16). 
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Theorem 5.4. Let F = (Fj, 1 < i < d) £ Td- 

(a) If there exists 1 < i < d such that H{Fi) = —oo, or if I{F) = +oo, then we have 
maxj 7 g£OS(p) H{F) = —oo. 

(b) If H{Fi) > —oo for all 1 < i < d, and JI(F) < +oo, then we have F G 
maxpg£OS(p) II{F) > —oo, and Fp defined in (53) is the unique cdf in T^'^(F) such 
that F(Fp) = maxpg£OS(p) i4(F). Furthermore, the density funetion of Fp exists, 
and is given by /p defined in (55). We also have: 

d 

H{Ff) = d-l + Y, H{F,) - J(F). 

i=l 


Proof The proof of case (a) is postponed to Section 6.7. 

We shall assume that II{Fi) > —oo for all 1 < i < d and < +oo. This implies that 

the densities fj of Fj exist for 1 < i < d and, thanks to Lemma 3.17, that F G Fj). Let Fp 
be defined by (53), that is the cdf with copula Cp from Proposition 5.1 and one-dimensional 
marginals cdf F. Thanks to Proposition 5.1, we have Fp G F^'^(F). 

We deduce from (15), Propositions 3.13 and 3.21, Theorem 4.7 case (b) and Proposition 
5.1 that Fp is the only cdf such that i7(Fp) = maxpg^osj-p) 11(F). We deduce from (15), 
(51) and Lemma 3.14 that: 

d 

F(Fp) = d - 1 + ^ H(Fi) - J(F). 

i=l 

Since the copula Cp is absolutely continuous with density cp given in (50), we deduce from 
(53) that Fp has density /p given by, for a.e. x = (xi, ...,, x^) G 


d 

(56) /p(x) = cp(Fi(xi),...,Frf(xrf)) JJfj(xi). 

i=l 

Recall the expression (50) of cp as well as FTj defined by (47), for 1 < i < d. Using the change 
of variable s = G~^(t) and (24), we get (similarly to the proof of Lemma 3.14); 

(57) Kio67}oFi(xi)-KioS7.('_^,oFi_i(xi-i)= £i(s)ds. 


Using (24), we also get: 


(58) 


fi(Xi) 


%-i) o o Ffixi) - Ffixi) 

According to (6), we have fi(t)dt-a.e. that F)"^ 
that 0*=! ^i(xi) dxi ■ ■ ■ dxd-a.e.: 


_ fijxi) _ 

Fj_i o FG^ o Ffixi) - Ffixi) 

oFi(t) = t. For 1 < i < d, we have from (6) 


(59) 




We deduce from (50), (56), (57), (58), (59) and Lemma 5.3 that a.e. for x 

/p(x) = fi(xi) jj£i(xi)e 1 lf(x). 

i=2 


(xi,...,Xd) G M"*: 


□ 
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Remark 5.5 
if F = 

S (ZW 


Fi,l 


We deduce from the proof of Theorem 5.4 case (b) and Proposition 5.1, that 
< i < d) £ then /f defined by (55) is a probability density function on 


Remark 5.6. The density /f has a product form on , that is it can be written as, for a.e. 

X = (xi,... ,Xrf) G 

d 

( 60 ) /f(x) = ]Jpi(xi)liF(x), 

i=l 

where the functions (pi, 1 < i < d) are measurable and non-negative. 

In addition to Remark 5.6, the next Corollary asserts that Tp is the only element of 
£^‘^(F), whose density has a product form. 

Corollary 5.7. Let F G Let F G C^^{F) be an absolutely continuous cdf with density f 
given by, a.e. for x = (xi,... ,Xd) G /(x) = 0^=1 hi{xi)l]^F{x), with hi, 1 < i < d some 
measurable non-negative functions on M. Then we have F = Ff on 


Proof. Let X = (Xi,..., Xd) be an order statistic with cdf F, and F^^^ the cdf of Xu given 
by (17), with 11 uniform on Sd and independent of X. Then the cdf F^^^ is also absolutely 
continuous, and its density f^v^ is given by : 

1 

( 61 ) r^ix) = ^Jlh,{x^,))lLF{x^^), 

i=l 

where x^^ is the ordered vector of x. The one-dimensional marginal cdf’s of Xu are all equal 
to G given by (18). Let C G C‘^‘^(F) nC° denote the copula of F. Then according to (8), the 
copula S'f(C') of Xu is given by, for a.e. u = (tti,..., Ud) G 

5F(C)(n) = F*2'™(G-n^^)) • 


Therefore its density sf(C') can be expressed as: 


s.iC)(u)= /;-(G-‘(u)) TTl„„c-(„,>0, 




))> 0 }> 


where g is the density of G. Notice that for x = (xi,..., Xd) G S, we have a.e.: 

with fi the density of Xi. Therefore by Lemma 5.3 and since G is continuous, we have that 
nf=i hi o G~^{u(^i)) dui... dud-a.e.: 

1lf{G-\u‘^^)) = o 5(1) (m(i) ),..., od(rf)(n(d))). 


By Lemma 3.12, 5f(C') belongs to C^p and thus sf(C') = 0 a.e. on Z^f defined by (39). Then 
use (4) and (6) to get that 6^-^ o 5(j)(ri(j)) = ri(j) a.e. on Z^p. This gives: 

1j^f(G-^(u^^)) = = 1l,p(u) 
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that is sf(C') is of the form sf(C')(u) = (l/d!)!^^^ (u) 0^=1 some measurable non¬ 

negative functions (hi, 1 < i < d). Then, thanks to Proposition 4.6, we get that S'f(C') = 
Then, use Proposition 5.1 to get that F = Fp. □ 

6. Proofs 

6.1. Preliminary notations for the optimization problem. Recall notations from Sec¬ 
tions 2 and 3. In particular if rt = (rii,... ,Ud) G then ... ,rt(d)) denote the 

ordered vector of u. 

In order to apply the technique established in [5], we introduce the linear functional A = 
{Ai, 1 < i < 2d) : —)■ as, for / G and r ^ I : 

A(/)(?^) = /(«)!{„,<^} du and Ad+i{f){r) = f{u)^{u^i^<r} du for 1 < i < d. 

Let 5 = ((5(j),l < i < d) G be a multidiagonal, see Definition 3.6. We set = (6*,! < 
i < 2d) given by bi = id/ the identity function on I and bd+i = d(j), for 1 < i < d. If, for 
c G we have Ai{c) = bi, 1 < i < d and c > 0 a.e., then we deduce that c is the density 

of an absolutely continuous copula, say C. If we further have Ad+iic) = bd+i, for 1 < i < d, 
then 5 is the multidiagonal of C. 

Lemma 6.1. Let 5 and b^ = {h,! < i < 2d). If c G L^{I'^) is non-negative, symmetric 
and satisfies Ad+iic) = bd+i for 1 < i < d, then c is the density of a copula with multidiagonal 

5. 

Proof. The symmetry and non-negativity of c as well as the condition .4,^+1 (c)(1) = fjd c = 
6(i+i(l) = 1 ensures that c is a density function of an exchangeable random vector V = 
(Pi,..., Vd) on I^. Recall = (V(i), • • •, l^(rf)) denotes the corresponding order statistics. 
By symmetry, the lemma is proved as soon as we check that .4,1 (c) = bi. We have for r G I: 

d d .. 

Aiic)ir) = P(Pi < r) = j;P(P(q < r|Pi = P(,))P(Pi = P(q) = ^^5(0(0 ^ = L 

i=l i=l 

where we used the exchangeability of V and the definition of for the third equality, and 
(22) for the last. This gives .4i(c) = 6i. □ 

6.2. Proof of Proposition 4.5. Let 6 G . Lemma 3.15 implies that d(i)(('I'^)‘^) has zero 
Lebesgue measure for all 2 < i < d with given by (30). By construction, the function 
defined by (45) is non-negative, symmetric and well defined a.e. on I^. Recall the notation 
{g\^\d^P) used in (40). We define the functions on / as, for 1 < i < d -|- 1, t G ig\^\d^p) 
(with the conventions Tf = (0, di), = [gd+i, 1)): 

(62) Bd+i{t) = 1 and Bfit) = J ai{s)Bi+iis) ds for 1 < i < d. 

For t G ('kf)^, we set Ri(t) = 0. Recall Ki defined in (47) for 1 < i < d -|- 1 with the 
convention Kd+i = 0. We show that Bi can be simply expressed by Ki on 

Lemma 6.2. Let 1 < i < d + 1 and t G Then we have: 

(63) Bfif) = exp i-Kfit)) . 
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Proof. For i = 
that = 




d + 1, the result is trivial. We proceed by induction on i. 
exp holds for some 1 < i < d, and all t G V’f+i- 


We suppose 
We have for 


B^{t) 



ai{s)Bi+i{s) ds 



K'iis) I'i'fn'i'f+i ('S) ds. 

K'{s)e-^ds) dg 


Jt 

exp{-Ki{t)), 


where we used the definition of a* given by (46) for the second equality, the induction hy¬ 
pothesis for the third equality, {t,d[^'^) C 'I'f and Lemma 3.15 for the fourth equality, and 
finally Remark 4.4 for the fifth equality. This ends the induction. □ 


Similarly, we define the functions Ei on I as, for 0 < i < d as for t G 


yyi+l’^i+l) 


(64) 


Eo{t) = 1, and Ei{t) 


ai{s)Ei_i{s) ds for 


1 < f < d. 


For t G ('I'^_(_j^)'^ we set Ei(t) = 0. The next Lemma gives a simple formula for Ei on 
Lemma 6.3. Let 0 < i < d and t G 'kf+i- Then we have: 

(65) Ei{t) = (d(i)(t) - d(i+i)(t)) exp (Ki+i(t)) . 

Proof. For i = 0 the result is clear thanks to the convention d^o) = 1 and (48). We proceed 
by induction on i. We suppose that Ei^ift) = (d(j_i)(t) — d(j) (t)) exp(iLj(t)) holds for some 
1 < i < d, and all t G V’f- Let us denote hi = d(j_i) — d(j). Before computing Ei{t) for 
t £ ( 5 ^+^)), we give an alternative expression for exp(iLi(s)) for s G 


^Ki+ps) ^ gjj-p 


* h'i+iju) 
hi+i{u) 


+ 


M+i 


(u) 


( 66 ) 


(t) 

hi+i{s 


■ exp 


K+iiu) 


x+l 


hi+i{u 

j 


+ 




hi+i{u] 


du 


+ 


5[i){u) 


nrSdf /lj_l_l('u) 

*+l 


du 
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Then we have for t G 

i{t) = I ^ ai{s)Ei_i{s) ds 

Ja^A\ 

/(.) K'iis) hi{s) is)ds 

O'.' I 1 


'9'i+i 

rt 




'9i+i 

rt 


= / ,, (s) Ivjy^nvi/^^, (s) ds 

d9i+i 


i i+1 


rt h' 


= hj+i(f)exp - 


du 


= /ii+i(f)exp y- 
= hi+i{t) exp{Ki+i{t)) 


hj+iiu) 

1^1 hi+i(u) 


>9^, 


(M± 

\hi+i{s) 


exp 




4^1 di+iiu 


■ du \ ds 


hi+i (u) 


du 


where we used the definition of ai given by (46) for the second equality, the induction hy¬ 
pothesis for the third equality, Lemma 3.15 and (66) for the fifth equality, and for the seventh 
equality we use that, for t G (similarly to Remark 4.4): 


/ 

J rt 


* <^(i)(s) 


■ ds < 


I 




ds = log 


d(i)(t) -%+i)(5l+\ 


hi+i{s) V%)(’^Si) - %+i)(ffm) 


giving lim,. (,) I (,) , , ds = —oo. 

t\gr+i'^^r+i 


The following Lemma justifies the introduction of the functions Bi,Ei 
Lemma 6.4. ITe have with U(o) = 0 ft>f 1 < f < d, t G T^.- 


□ 


(67) 


C5iu)l{u^._^^<t<u^i^} du = Bi{t)Ei_i{t). 


Proof. The dehnition (62) of Bi for 1 < i < d gives that for t £ I: 


d-l 


Bi{t) — I ai(ri)ai+i(ri+i) . . . (td{t'd)^{t<ri<ri+i<...<rd<l}^{[t,ri)C'i>f} rr ^{(^iAj+l)C'I'L il dt': 


■J=l 


with r = (ri,ri+i, ... ,rd) G Similarly, we have for 1 < i < d, t G / that Ei-i{t) is 

equal to; 


i-2 


«l(9l)«2(g2) • • • «'*-l(9*-l)l{0<gi<g2<...<<?i-i<t}l{fe_i,t]c^f} 11 dq, 

i=i 
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with q = {qi,q 2 , ■ ■ G P Multiplying Bi{t) with Ei_i{t) gives; 




{t)= 

.7 = 1 




= / Waj{uj)l{u^_^<t<u^}'^L^{u)du 
j=i 

= d\ csiu)l{ui_i<t<ui} du 
JA 

where we used the symmetry of cs for the fourth equality. □ 

Lemma 6.4 with i = 1 ensures that cs{u) du = lim^^o- 6 i(^)-E'o(^) = I, that is cs a 
probability density function on Now we compute Ad+i{cs)- We have, for t G iLf: 

Ad+i{cs)it) = du = l- Bi{t)Eo{t) = 

where we used Lemma 6.4 with i = 1 for the third equality, then (63) and (48) for the fourth 
equality. By continuity this gives ^^+ 1 ( 05 ) = on I. For 2 < i < d, we have by induction 
for t £ 

Ad+i{cs){t) = C5(«)l{«(i)<i} du 

= du - q(«)!{«(,_!)<*<«(,)} du 

= Ad+i-i{c5){t) - Bi{t)Ei_i{t) 

= %-i)(t) - (%-i)(t) - %)(t)) 
d{i) (t); 

where we used the induction and Lemma 6.4 for the third equality, as well as (63) and (65) 
for the fourth. By continuity, we obtain Ad+i{cs) = (5(j) on I. Then use Lemma 6.1 to get 
that Cs is the density of a (symmetric) copula, say Cs, with multidiagonal <5. 

The computation of H{Cs) is technical. By symmetry of Cs, we have: 

H{Cs) = -d\ [ cs logics). 

JA 

Let e G (0,1/2). We define for 1 < i < d the set If = {t £ E, [t — £,t + e) <Z r\ 
as well as Ig = (e, di — e) and Idj^i = {gj+i + e, 1 — e). We set = A n 0^=1 df- Since 
xlog(x) > — 1/e for X > 0, we deduce by monotone convergence that: 

( 68 ) H{Cs) = \lcnHe{Cs), 

e\.0 


HeiCs) = -d\ / C5log(c5). 
J A^ 


We can decompose H^iCs) as: 


HeiCs) — di(e) + J2(£) + Jsi^) + «^4(£)) 
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with: 


Ji{e) = cs{u)log{6[i^{ui)) du, 

d n 

J 2 (e) = d\'^ I C 5 (w)log ((5(j_i)(wi) - 5(i)(iXi)) du, 
i=2 

d n 

■hie) = dl'^ cs{u) {Ki{ui) - Ki{ui-i)) du, 


i=2 


J 4 (e) = log((i!) dl / cs{u)du, 

Ja^ 


where we used (48) to express ai = a.e., so that the sums in J 2 and J 3 start at i = 2. 

Since Ad+i{cs) = 5(i), we deduce that for 1 < i < d and any measurable non-negative 
function h defined on I: 

(70) j ^cs(u)h{u(i))du = j 5'^j^(t)h(t)dt. 

In particular, we get: 



log(d'(.)(U(i))) 


du 


d[i)(t) |log(<5(.^(t)) 


or 


dt, 


which is finite thanks to Remark 3.8. Therefore, by the dominated convergence and using 
(70) again, we have: 


lim Ji(e) = '^H(5(^i)). 


2=1 


For J 2 (£)) we notice that the integrand is non-positive a.e., since for t £ 1, 2 < i < d, we 
have (5(j_i)(t) — d(i)(t) < 1. Therefore, we get: 


liin J2(e) 
£—^0 


d 

-d!^ / C 5 (u) |log(d(i_i)(ui) - d(i)(ui))| du =-JI(d), 


i=2 


where we used monotone convergence for the first equality and (70) as well as the dehnition 
(16) of JI(d) for the second. 

We now compute Jsis). For t G I^+i^ ~ ^ ^ EQ(t) = 1. For 

1 < i < d, we dehne for t G (g^i\d^P) and t' G dj:^\): 

^max((Ipl —£,I) i-t' 

Bl(t)= ai(s)Rf+i(s)ds, and El(t')= ai(s)El_-^(s) ds. 

Jt dmin(g>;()j^-|-£,t') 

By monotone convergence, we have for t G and t' G 'kf+i: 

lira Bf(t) = BAt) and \iraEl(t') = EAt'). 

£->■0 £^0 
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An integration by parts gives; 
d 


J^{e) = d\'^([ cs{u)Ki{ui)du- [ cs{u)Ki{ui-i) du 
^ \Ja^ Ja^ y 

= '^( [ Ef_^{t)ai{t)Ki{t)Bl^^{t)dt- f Ef_2{t)ai-i{t)Ki{t)Bf{t)d^ 

i =2 •^E-l J 

+ Y( (t) dt - [ Ef_^{t)Ki{t)ai{t)Bf^^{t) d^ 

i =2 vE E? J 

d „ 

= Y E!_,{t)K'{t)B!{t)dt. 

i =2 Je 

By monotone convergence and thanks to (47), (63) and (65), we get; 

d „ d „ 

lim J3(e) = Y Ei_i{t)K'i{t)Bi{t) dt = 'Y = {d-l). 

i =2 i =2 E 

We deduce from (68) and (69) and the limits for Ji, J 2 , J 3 and J4 as e goes down to 0 that; 

d 

H{Cs) = Y + {d-l)+ log(d!). 

i=l 


6.3. The optimization problem. Let 5 G 2?®. Recall notation from Section 6.1. The 
problem of maximizing H over can be written as an optimization problem {P^) with 
infinite dimensional constraints; 


(P^) 


maximize H{c) subject to 


.4,(c) = , 

c > 0 a.e. and c G L^(/‘^). 


Notice that if / G is non-negative and solves A{f) = 6^, then / is the density of a 

copula. We say that a function / is feasible for (P^) if / G / > 0 a.e., A{f) = and 

H{f) > —00. We say that / is an optimal solution of (P^) if / is feasible and H{f) > H{g) 
for all g feasible. The next Proposition gives conditions which ensure the existence of an 
optimal solution. 


Proposition 6.5. Let 6 G P°. If there exists c feasible for (P^), then there exists a unique 
optimal solution to (P^) and it is symmetrie. 

Proof. Since A{f) = b^ implies .4i(/)(l) = 6i(l) that is Jjdf(x)dx = 1, we can directly 
apply Corollary 2.3 of [5] which states that if there exists a feasible c, then there exists a 
unique optimal solution to (P*^). Since the constraints of (P*^) are symmetric, such as the 
functional H, we deduce that if c* is the optimal solution, then so is c* defined for n G Sd 
and u G as c*{u) = c*(tt7r). By uniqueness of the optimal solution, we deduce that c* = c* 
for all permutations vr G Sd] hence c* is symmetric. □ 


Combining Lemmas 4.2 and 4.1 gives the following Corollary on the support of any c 
verifying .4.(c) = b^. 



MAXIMUM ENTROPY DISTRIBUTION OF ORDER STATISTICS WITH GIVEN MARGINALS 


27 


Corollary 6.6. Let 6 G If c G is non-negative and verifies A{c) = , then c = 0 

а. e. on with Ls defined by (41) and \ Ls- 

б. 4. Reduction of the optimization problem (P^). Let S G Since the optimal 

solution of (P'^) is symmetric, see Proposition 6.5, we can reduce the optimization problem 
by considering it on the simplex A. We define fi to be the Lebesgue measure restricted to 
(Z^n Ls) n A: fj,{du) = We define, for / G L^{I^): 


= f (u) log{ f (u)) fi{du). 

Jid 

From Corollary 6.6 we can deduce that if c G is non-negative symmetric and solves 

.4,(c) = 6'^, then: 

(71) H{c) = d\H>^{c). 

Let us also define, for / G L^{I'^), 1 < i < d, r £ I: 


■ 4 f(c)(r) = d! [ c{u)l{ui<r}Kdu)- 

Jjd 


We shall consider the restricted optimization problem (P^) given by: 



maximize H^{c) subject to 


A^{c) = <5, 
c > 0 ^-a.e. and c G 


We have the following equivalence between (P^) and (P))). Recall vP^ denote the ordered 
vector of tt G 


Corollary 6.7. Let 6 G If c is the optimal solution of (P^) then it is also an optimal so¬ 
lution to (P^). If c is an optimal solution of (P^), then c, defined by c(u) = c(u^^)lz^nLs(u) 
is the optimal solution to (P^)■ 

Notice the Corollary implies that {Pffi has a /U-a.e. unique optimal solution: if ci and C 2 
are two optimal solutions of {Pffi then ;U-a.e. ci = C 2 . Thanks to Proposition 6.5 and (71), 
Corollary 6.7 is a direct consequence of the following Lemma that establishes the connection 
between the constraints. 


Lemma 6.8. Let 5 G P°. For c G Lf{I^) symmetrie and non-negative the following two 
conditions are equivalent: 

(1) Me] = 

(2) A^{c) = 5 and c = 0 a.e. on Zs[j L^. 

Proof. Assume that A{c) = b^. We have, by Corollary 6.6, that c = 0 a.e. on Zs U L^. This 
and the symmetry of c gives, for 1 < i < d, r £ I: 

■4f(c)(r) = d\ c{u)l{u^.^<r}lAiu) du = j c(u)l{„^.j<^} du = d(i)(r). 

On the other hand, let us assume that A^(c) = S and c = 0 a.e. on Zs U L^. We have, for 
1 < i < d, r £ I: 

Ad+i(c)(r) = / c(u)liu(i)<r}Iz^nLs(u) du = d! / c(u)l^ui<r} h(du) = d(i)(r), 

JJ 
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where we used c = 0 a.e. on for the first equality, the symmetry of c and the dehnition 

of ^ for the second, and A^{c) = 5 for the third. Lemma 6.1 ensures then that Ai{c) = bi for 
1 < i < d. This ends the proof. □ 


6.5. Solution for the reduced optimization problem {P^)~ Let 5 G T)^. We compute 
{A^^)* : L°°{IY the adjoint of A^^. For A = (A*,! < i < d) G L°°{lY and 

/ G we have: 


((^^)*(A),/) = (A,^'"(/)) = V [ \i{r)[ f{u)ls^u^<r}d^i{u)dr= [ f (u)Ai{ui) d^{u), 

1^1 -Ji JP JP 1^1 

where we used the definition of the adjoint operator for the first equality, Fubini’s theorem 
for the second, and the following dehnition of the functions (Aj, 1 < i < d) for the third: 

Ai{t) = j \i{r)l{r>t} dr, tG I. 

Thus, we have for A G L°°{lY and u = (tti,..., Ud) G 


(72) 


d 

(.4'‘)*(A)W = ^A.(«.), 

i=l 


We will use Theorem 2.9. from [5] on abstract entropy minimization, which we recall here, 
adapted to the context of {Pf^)- 

Theorem 6.9 (Borwein, Lewis and Nussbaum). Suppose there exists c > 0 fi-a.e. which is 
feasible for {P^)- Then there exists a p-a.e. unique optimal solution, c*, of {P^). Further¬ 
more, we have c* > 0 pi-a.e. and there exists a sequence (A"',n G N*) of elements of L°°{lY 
such that: 


(73) / c*(u)|(.4^)*(A")(n)-log(c*(n))| ^iidu) -^ 0. 

J Id n^oo 

Now we are ready to prove that the optimal solution c* of (P^) is the product of measurable 
univariate functions. 


Lemma 6.10. Let 6 G . Suppose that there exists c > 0 p,-a.e.whieh is feasible for (P^). 
Then there exist non-negative, measurable funetions {a*, 1 < i < d) defined on I such that 
a*{s) = 0 if 5'^-Ys) = 0 and the function c* defined a.e. on by: 

d 

c*{u) = j^'^Ls{u)X\a*{ui) 
i=\ 

is the optimal solution to {P^). 

Proof. According to Theorem 6.9, there exists a sequence {X^,n G N) of elements of L°°{lY 
such that the optimal solution, say c*, satishes (73). This implies, thanks to (72), that there 
exist d sequences (A”,n G N*,l < i < d) of elements of L°°{I) such that the following 
convergence holds in L^[I^,c*pl): 

d 

(74) ->log(c*(w)). 

^ ^ n—^C)0 
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We first assume that there exist Aj, 1 < i < d measurable functions defined on I such that 
lj,-a.e. on S: 


'^Ai{ui) = log(c*(n)). 


Set a* = v^exp(Aj) so that ^-a.e. on S: 




Recall fj,{du) = l(zinLs)nA{u) du. From the definition (39) of Zs, we deduce that without 
loss of generality, we can assume that a*{ui) = 0 if = 0. Therefore we obtain 

c*(u) = (l/dl)lLs(u) Ui=i for ^ e 

To complete the proof, we now show that (75) holds for Aj, 1 < i < d measurable functions. 
We introduce the notation U(_j) = (ui,..., Ui_i, «j+i,..., Ud) G . Let us define the prob¬ 
ability measure P{du) = c*{u)fi{du)/ Jjd c*{y)^{dy) on We fix j, 1 < j < d. In order to 
apply Proposition 2 of [17], which ensures the existence of the limiting measurable functions 
Aj, 1 < i < d, we first check that P is absolutely continuous with respect to Pf <8* 
where Pl{du(^_j'^) = P{du(^_j-^duj) and P^iduj) = P(du(_jjduj) are the 

marginals of P. Notice that there exists a non-negative density function h such that P{du) = 
h{u(^_j-j,uj)du(^_j)duj. Let /ii(tt(_j)) = J h{u(^_j-j,uj)duj and h 2 {uj) = //i(tt(_j), Uj)dtt(_j) 
denote the density of the marginals P( and P^. Then the density of the product mea¬ 
sure P/ (8i Pj is given by P( 0 P^idu) = hi{u(^_j^)h 2 {uj)du(^_j^duj. The support of the 
density h is noted by Tq = {u G I^]h{u) > 0}, and the support of the marginals are 
noted by Ti = {u G /‘^“^;/ii(u) > 0} and T 2 = G /;/i 2 (t) > 0}. With this nota¬ 
tion, we have that a.e. Tq C Ti x T 2 (that is Tq n (Ti x T 2 Y is of zero Lebesgue mea¬ 
sure). li A <Z I'^ is such that f lA(u)hi(u(_j))h 2 (uj)du(_j)duj = 0, then we also have 
/ ^An{TixT 2 ){''^)hi{u(-j))h 2 {uj)du(^-j)duj = 0. Since / 11/12 is positive on Ti x T 2 , this implies 
that A n (Ti X T 2 ) has zero Lebesgue measure. Therefore we have: 

j lA{u)h{u)du = j lAn{TixT2)iu)hiu)du + j lA\(TixT2)iu)hiu)du = 0 , 


since h = 0 a.e. on A \ (Ti x T 2 ). This proves that P is absolutely continuous with respect 
to Pf 0 P^ Then according to Proposition 2 of [17], (74) implies that there exist measurable 
functions and Aj defined respectively on and I, such that Py-a.e. on A; 

log(c*(n)) = + Aj{uj). 

As y-a.e. c* > 0, this equality holds y-a.e. on S. Since we have such a representation for 
every 1 < j < d, we can easily verify that log(c*(rt)) = Yli=i y-a.e. with Aj = Aj up 

to an additive constant. 

□ 


6.6. Proof of Proposition 4.6. Let S G P®. Recall that denotes the ordered vector 
of rt G M'^. Let c be the density of a symmetric copula in such that A{c) = and c is of 
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product form, that is, thanks to Corollary 6.7, c{u) = with 


1 

c*(u) = ^ lLsiu)Y[a*{ui), u = (ui,...,Urf) G A, 


2=1 


where a*, 1 < i < d are measurable non-negative functions defined on I. In this section, we 
shall prove that c equals cs defined by (45); that is, for all 1 < i < d, a* is a.e. equal, up to 
a multiplicative constant, to a* defined in (46). This will prove Proposition 4.6. 

Recall the definitions of 5 ) , m) , d) ^ from Section 4, for 1 < i < d -f 1. We deduce from 
(44) that; 

1 

c*{u) = — lLs{u)Y[a*{ui)l^s^^s (ui), u = {ui,...,Ud) G A. 


2=1 


We deduce also from Lemma 6.8 that A^{c*) = 5. We introduce the following family of 
functions; 

and for 1 < i < d, t G {g^i\d^p) and t' G (ffi+ij^+i)' 

/.rfW I't' 

Bt{t)= / a*{s)B*^,{s)ds, El{t')= a*{s)EU{s) ds. 

Recall the functions Rj, for 1 < z < d -f 1, and Rj, for 0 < z < d defined by (62) and (64). 
We will prove by (downward) induction on z G {1,..., d -f 1} that; 


(77) 


Bt{t) = Bt{mf)B,{t), t G (5?'\4'^)- 


For z = d -f 1, it trivially holds. Let us assume that (77) holds for z -|- 1, d > z > 1. Recall 
the convention Kd+i = 0, d(d+i) = 0 and d(o) = 1. Arguing as in the proof of Lemma 6.4, we 
deduce from A.f(c*) = (5(j) that for r G 'kf; 

dii){r) = d\[ c*{u)l{u <r}g{du) 


I Jd 


= d\ c*(«)l{«(i+i)<r} g-idu) + iLsiu) ^{Ui<r<ui+i} du 

= %+i)(r)+R;+i(r)R;(r). 

This gives on 'k^; 


(78) 


%) - %+i) = Bt+iE*. 


Notice (78) holds for i = d thanks to the conventions. We get on 

4 ,) - 4,+i) = -KU,B*^iE* + B:^,a*El, = + Bt^,a*EU. 

where we took the derivative in (78), twice the induction hypothesis for B*_^_^ and (63) for 
the first equality; then (47) and (78) for the second. We deduce that on 
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On we can divide (79) by (78) and get, thanks to (47): 


B* 


5', 


(i) 


%-l) - %) 


= K'. 


Notice that (B*)' = —a*B*_^_^. So using the representation (63) of Bi, we get that (77) holds 
for i. Thus (77) holds for 1 < i < d + 1. Then use (77) as well as (B*)' = —a*B*_^_^ and 

B'- = -UiBi+i to get that for t G (s'F^d-'’^) O ( 5 !+^, ^1+^): 


^r+i(K;i) 


Therefore if u = (ui,..., Ud) G , we have; 


d 

i=l 


^i(mi) 

Bd+iirrid+i) 


d 

i=l 


since when u G Ls, U(i_i) and U(j) belong to the same interval {g^^\ d^P) for 2 < i < d. This 
ensures that cs and c* are densities of probability function which differ by a multiplicative 
constant, therefore they are equal. This ends the proof of Proposition 4.6. 


6.7. Proof of case (a) for Theorems 4.7 and 5.4. We first consider the case d = 2. Let 
S with JI(d) = + 00 . Recall JI(d) is defined by (16). We have; 

= - Ji d( 2 ) (t) log(2(t - d( 2 ) (t))) dt 

= - log(2) - J^log{t - 6(2)(t)) dt + - 6(2)(t)) log(t - 6(2)(t)) dt 

= - log(2) - ^ log(t - d(2) (t)) dt + [{t - 6(2) (t)) log{t - 6(2) (t)) - (t - 6(2) (t))] J 
= - log(2) - log{t - 6(2) (t)) dt, 

where we used + 6 ( 2 ) = 2 t for the first equality, d( 2 )(l) = 1 and d( 2 )( 0 ) = 0 for the second 
and last. In particular, we obtain that JI(d) is equal to — log(2) + J{ 6 ( 2 )), with J' as also 
defined by (1) in [6]. Therefore we deduce case (a) of Theorem 4.7 (for d = 2) from case (a) 
of Theorem 2.4 in [6]. Then, we get from (15) and Theorem 4.7 case (a) that H{F) = —00 
for all F G /l^'^(F). This proves case (a) for Theorem 5.4 (for d = 2). 

We then consider the case d > 2. Let 6 G P® with JI(d) = + 00 . This implies that there 
exists 2 < i < dsuchthat fj 6 (-)(t) |log(d(j_i)(t) — d(j)(t))| dt = + 00 . Set F = (d(j_i), d(j)) and 

notice that F belongs to F 2 as 6 (i) is d-Lipschitz. Since fj 6 (-((t) |log((5(j_ i)(i) - <^(i)W)| dt = 
+ 00 , we deduce from the first part of this Section that H{F) = — 00 . 

Consider a copula C belonging to Cs H and U a random vector on with cdf C. 
According to Lemma 3.20 and Lemma 3.3, as C is symmetric, we have; 

= H{U) - log(d!) = H[C) - log(d!). 

It is easy to check that if A = (Ai,..., A^^) is a random vector on and 2 < i < d, then we 
have dd((Aj_i, Aj)) > H{X). This implies that, for V = (t/f^'f, t/f*'^), 

H{V) > H{C) - log(d!). 
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Since the cdf of is as C G Q, we deduce the cdf of V belongs to and thus 

H(y) = —oo. This implies that H{C) = —oo. Thanks to Proposition 6.5 which states that 
the entropy is maximal on symmetric copulas, we deduce that: 

max i7(C) = max HiC) = —oo. 
ceCs ceesC]C‘>y^ 

This proves cases (a) for Theorem 4.7. Then, we get from (15) that H{F) = —oo for all 
F G This proves case (a) for Theorem 5.4. 

6 . 8 . Proof of Theorem 4.7, case (b). Let 5 £ V with JI(5) < +oo. Thanks to Lemma 
3.15, J((5) < +00 implies that (5 G V^. By construction, cs introduced in Proposition 4.5 
verifies jj. — a.e. cg > 0. The density cg is a feasible solution to the problem (P^)- Theorem 
6.9 ensures the existence of a unique optimal solution c*. Furthermore, by Lemma 6.10, we 
have that there exist non-negative, measurable functions a*, 1 < i < d, such that c*{u) = 
{l/dl)lLs{u) l\i=l a*(ui) fi-a.e. By Corollary 6.7, the optimal solution c of (F^) is given by, 
for u = (ui,.. .,Ud): 

1 1^1 

c(u) = c*(u^^)lzcnLsiu) = - 1 l,(u) 

i=l 

Since c is of product form. Proposition 4.6 yields that c = cg a.e., therefore Cg is the unique 
copula achieving H{Cg) = maxcec^ HiC)- 
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7. Overview of the notations 

- J-d- set of continuous one-dimensional marginals cdf F = (Fi,..., of d-dimensional 
order statistics, see (11). 

- set of continuous one-dimensional marginals cdf F = (Fi,..., F^) of d-dimensional 
abs. cont. order statistics, see Definition 3.16. 

- Cd'- set of all cdf’s on 

- set of cdf’s on with continuous one-dimensional marginals cdf. 

- set of absolutely continuous cdf’s on R'^. 

- set of cdf’s of d-dimensional order statistics with continuous one dimensional 
marginals cdf. 

- /l^'^(F): set of cdf’s of d-dimensional order statistics with marginals cdf F, see (12). 

- set of symmetric cdf’s on R'’^. 

- Sy'- symmetrizing operator on copulas, associated to the marginals cdf F, see Defini¬ 
tion 3.1. 

- C: set of all copulas. 

- set of absolutely continuous copulas. 

- C'^'^(F): set of copulas of order statistics with marginals cdf F, see (13). 

- set of symmetric (permutation invariant) copulas. 

- image of the set C'^'^(F) by the operator 5 f, see (20). It is the set of 
symmetric copulas with multidiagonal 5^. 

- C 5 : set of copulas with multidiagonal <5, see Section 3.2. 

- C^: set of abs. cont. copulas with multidiagonal d, see Section 3.2. 

- V: set of multidiagonals of copulas, see Section 3.2. 

- V^: set of multidiagonals of abs. cont. copulas, see Section 3.2. 

- 'hf: set of points t G R for which the marginals cdf F = (Fi,... ,Fd) verify Fj_i(t) > 
Fi(t) , see (30). 

- set of points u = (ui,... ,Ud) G for which F]"^(ui) < ... < F^^(urf), see (34). 
The density of all copulas in C‘^'^(F) vanishes on T^. 

- L^: set of ordered vectors x G R'’* such that the marginals cdf’s F = (Fi,... ,Fd) 
verify Fj_i(t) > Fj(f) for all t G (xj_i,Xi), 2 < i < d, see (52). The density of any 
abs. cont. cdf in £^‘^(F) vanishes outside . 

- Ls'. set of points u = {ui,... ,Ud) G for which all points t G (u(j_i),U(j)) verify 
(I(j_i)(t) > (5(j)(t) for all 2 < i < d, see (41). The density of any copula in vanishes 
outside Ls. 

- Zs: set of points u = {ui ,..., Ud) G such that d(j)(it(i)) = 0 for some 1 < i < d, see 
(39). The density of any copula in vanishes on Zs- 



34 


CRISTINA BUTUCEA, JEAN-FRANgOIS DELMAS, ANNE DUTFOY, AND RICHARD FISCHER 


References 

[1] B. C. Arnold, N. Balakrishnan, and H. N. Nagaraja. A first course in order statistics, 
volume 54. Siam, 1992. 

[2] J. Averous, C. Genest, and S. C. Kochar. On the dependence structure of order statistics. 
Journal of Multivariate Analysis, 94(1):159-171, 2005. 

[3] P. J. Bickel. Some contributions to the theory of order statistics. In Proc. Fifth Berkeley 
Sympos. Math. Statist, and Probability (Berkeley, Calf., 1965/66), Vol. I: Statisties, 
pages 575-591. Univ. California Press, Berkeley, Calif., 1967. 

[4] P. J. Boland, M. Hollander, K. Joag-Dev, and S. Kochar. Bivariate dependence properties 
of order statistics. Journal of Multivariate Analysis, 56(l);75-89, 1996. 

[5] J. Borwein, A. Lewis, and R. Nussbaum. Entropy minimization, DAD problems, 
and doubly stochastic kernels. Journal of Funetional Analysis, 123(2);264 - 307, 
1994. ISSN 0022-1236. doi: http;//dx.doi.org/10.1006/jfan. 1994.1089. URL http: 
//www.sciencedirect.com/science/article/pii/S0022123684710895. 

[6] C. Butucea, J.-F. Delmas, A. Dutfoy, and R. Fischer. Maximum entropy copula with 
given diagonal section. Journal of Multivariate Analysis, 137:61 - 81, 2015. ISSN 
0047-259X. doi: http://dx.doi.Org/10.1016/j.jmva.2015.01.003. URL http://www. 
sciencedirect.com/science/article/pii/S0047259X15000081. 

[7] H. A. David and H. N. Nagaraja. Order statisties. Wiley Online Library, 1970. 

[8] B. V. de Melo Mendes and M. A. Sanfins. The limiting copula of the two largest or¬ 
der statistics of independent and identically distributed samples. Brazilian Journal of 
Probability and Statistics, 21:85-101, 2007. 

[9] D. Dubhashi and O. Haggstrom. A note on conditioning and stochastic domination for 
order statistics. J. Appl. Probab., 45(2):575-579, 2008. ISSN 0021-9002. doi: 10.1239/ 
jap/1214950369. URL http://dx.doi.org/10.1239/jap/1214950369. 

[10] T. Hu and H. Chen. Dependence properties of order statistics. Journal of Statistical 
Planning and Inference, 138(7):2214-2222, 2008. 

[11] P. Jaworski. On copulas and their diagonals. Information Scienees, 179(17):2863 
- 2871, 2009. ISSN 0020-0255. doi: 10.1016/j.ins.2008.09.006. URL http://www. 
sciencedirect.com/science/article/pii/S0020025508003836. 

[12] P. Jaworski and T. Rychlik. On distributions of order statistics for absolutely continuous 
copulas with applications to reliability. Kybernetika, 44(6):757-776, 2008. 

[13] S. Kim and H. David. On the dependence structure of order statistics and concomitants 
of order statistics. Journal of statistical planning and inference, 24(3):363-368, 1990. 

[14] R. Lebrun and A. Dutfoy. Copulas for order statistics with prescribed margins. Journal 
of Multivariate Analysis, 128:120-133, 2014. 

[15] J. Navarro and N. Balakrishnan. Study of some measures of dependence between order 
statistics and systems. Journal of Multivariate Analysis, 101(l):52-67, 2010. 

[16] J. Navarro and F. Spizzichino. On the relationships between copulas of order statistics 
and marginal distributions. Statist. Probab. Lett., 80(5-6):473-479, 2010. ISSN 0167- 
7152. doi: 10.1016/j.spl.2009.11.025. URL http://dx. doi . org/10.1016/j . spl. 2009. 
11.025. 

[17] L. Riischendorf and W. Thomsen. Note on the Schrodinger equation and /-projections. 
Statist. Probab. Lett., 17(5):369-375, 1993. ISSN 0167-7152. doi: 10.1016/0167-7152(93) 
90257-J. URL http: //dx. doi . org/10.1016/0167-7152(93) 90257-J. 

[18] V. Schmitz. Revealing the dependence structure between and Journal of 

statistieal planning and inferenee, 123(1):41-47, 2004. 



MAXIMUM ENTROPY DISTRIBUTION OF ORDER STATISTICS WITH GIVEN MARGINALS 


35 


[19] N. Zhao and W. T. Lin. A copula entropy approach to correlation measurement at the 
country level. Applied Mathematics and Computation, 218(2):628 ~ 642, 2011. ISSN 
0096-3003. doi: 10.1016/j.amc.2011.05.115. URL http://www.sciencedirect.com/ 
science/article/pii/S0096300311007983. 

Cristina Butucea, Universite Paris-Est, LAMA (UPE-MLV), 77455 Marne La Vallee, France. 
E-mail address: cristina.butucea@univ-mlv.fr 

jEAN-FRANgois Delmas, Universite Paris-Est, CERMICS (ENPC), 77455 Marne La Vallee, 
France. 

E-mail address: delmas@cermics.enpc.fr 

Anne Dutfoy, EDF Research & Development, Industrial Risk Management Department, 92141 
Clamart Cedex, France. 

E-mail address: cuine.dutfoy@edf.fr 

Richard Fischer, Universite Paris-Est, CERMICS (ENPC), 77455 Marne La Vallee, France, 
EDF Research & Development, Industrial Risk Management Department, 92141 Clamart Cedex, 
France. 

E-mail address: fischerr@cermics.enpc.fr 



